Automatic 3D face synthesis using single 2D video frame - Electronics Letters

Introduction: Automatic 3D face synthesis plays a crucial role in many applications. One example is real-time 3D model-based video coding. Here, a generic 3D face model should ideally be adapted automatically to a human face in the first video frame. Additionally, facial texture should be acquired without supervision. However, the time-consuming 3D scanner [1] is not accepted for such an automatic system. The method of using two orthogonal photos [2] cannot provide real-time processing either. Feng [3] synthesised the face only from a single image, but this method needs to estimate head rotation parameters by using another reference image. Moreover, it hardly brings accurate results in 3D face model adaptation by only using three facial features. Strictly speaking, there is no fully automatic 3D face synthesis from a single image proposed to make such a real-time video application a reality. This Letter presents an automatic synthesis system, enabling 3D face modelling from an arbitrary 2D header-and-shoulder video frame without partial occlusion.